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Abstract 



In this paper, the authors use data from interviews and observations in four urban 
elementary schools - two high and two low performing - to examine how schools 
respond to high stakes accountability policies. The authors argue that school responses to 
high stakes accountability depend on school context. In low-performing schools, 
responses focus narrowly on complying with policy demands, focusing on improving the 
performance of certain students, within benchmark grades, and in certain subject areas. 

In contrast, higher performing schools emphasize enhancing the performance of all 
students regardless of grade level and across all subject areas. Given the concentration of 
poor students and students of color in the lowest performing schools, the authors 
conclude that issues of educational equity need to be given careful consideration in the 
implementation of high stakes accountability policies. 
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One of the most consistent findings in educational research is that family 
background is linked to children's educational outcomes, attainment, and adult 
occupational status (Blau and Duncan, 1967; Coleman et al., 1966; Jencks et al., 1972; 
Orfield, 1993). While education is viewed by many as an important mechanism for social 
mobility - tied to the “common school” belief that all children should have equal 
educational access and opportunity (Cremin, 1951; Muller and Schiller, 2000) - many 
scholars argue that schools reproduce social inequality. Over the past decade, policy 
makers have mobilized an arsenal of policy instruments in an effort to ensure that all 
children receive high quality education. One increasingly popular but controversial 
strategy relies on external accountability mechanisms, including high stakes testing, to 
transform instructional practices and make teachers and students more accountable for 
their performance. Critics argue that these policies will exacerbate inequalities by leading 
teachers to marginalize low-performing students (Clotfelter and Ladd, 1996; McDill, 
Natriello, and Pallas, 1986) and causing teachers to teach these students only the material 
covered on standardized tests. Others contend that such policies are misguided because 
limited resources, unprepared teachers, and ineffective instructional practices rather than 
incentives are the problem that needs to be addressed (Darling-Hammond, 1994). 

Proponents argue that such assessments will reduce gatekeeping processes such as 
tracking and low teacher expectations that disadvantage certain students. External 
assessments, it is argued, provide objective information for school-based decision- 
making and therefore work against more subjective judgements that contribute to 
stratification (Muller and Schiller, 2000; Coleman, 1997). For example, supporters argue 
that teachers’ assessments of students’ ability as well as decisions about course placement 
and grouping arrangements inside classrooms could be based on more objective 
information from standardized tests (Muller and Schiller, 2000). 
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Given the increasing emphasis on external assessment and accountability, and the 
strong arguments on both sides of the issue, it is interesting that with a few notable 
exceptions (Muller and Schiller, 2000; Roderick, Bryk, Jacob et al., 1999) research on 
how these policies unfold in schools and the mechanisms through which they impact 
student learning has been limited. Moreover, few studies have explored how the 
implementation of these policies may be situated in certain school contexts and how this 
may influence their impact on students. 

The study that is reported in this paper examines the implementation of high- 
stakes testing in high and low performing schools. We seek answers to two related 
questions. First, is high stakes accountability policy perceived and implemented 
differently in high and low-performing schools? Second, does the implementation of high 
stakes testing in these schools suggest that it will reduce social stratification through the 
mechanisms outlined by testing proponents? 

High Stakes Accountability and Stratification 

One aim of accountability policies is to insure that all students receive high 
quality instruction and reach a certain level of competence in core subject areas (Muller 
and Schiller, 2000). Some districts, like Chicago, have adopted a high stakes version of 
these policies that link student performance on examinations to consequences for schools 
and, in some instances, students themselves. Opponents of these policies argue that for 
these approaches to be fair, instructional changes should precede consequences for 
students (Heubert and Hauser, 1999) and that such policies create incentives for 
marginalizing low performing students (Clotfelter and Ladd, 1996; McDill, Natriello, and 
Pallas, 1986). Neutral observers caution that those implementing such policies must 
insure the adequacy of educational resources for the tested students, attend to the 
reliability and validity of the exams for their intended purposes, and avoid basing 
decisions on one test (AERA 2000). 
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Proponents of these policies suggest that they can reduce inequality through 
increasing student motivation, creating incentives for teachers to seek improvements in 
student outcomes, providing more objective information about students’ performance for 
school based decision-making, and increasing academic press in schools - particularly 
those serving low-income and minority students (Coleman, 1997; Shouse, 1997; Muller 
and Schiller, 2000). 

Coleman (1997) advocates the development of output-driven schools, a key 
component of which would be external assessment and accountability. He argues that 
external assessments such as student performance tests would create new incentives for 
school improvement, providing objective information for teachers to assess students and 
make course placement decisions, thereby reducing the gatekeeping functions of schools 
(Coleman, 1997; Muller and Schiller, 2000). Shouse (1997) argues that designing more 
output-driven schools would also increase academic press and have particularly 
beneficial consequences for students with lower socioeconomic status. Therefore, 
proponents of these policies suggest that three key mechanisms - the creation of new 
incentives, the provision of objective information for school decision making, and the 
increase of academic press - will combine to reduce schools’ and teachers’ gatekeeping 
practices and contribute to a reduction in stratification. 

While arguments have been forwarded in support of and in opposition to these 
policies, surprisingly little research closely examines how they play out in schools. The 
data on outcomes that does exist presents a mixed picture. Data on the implementation of 
high stakes testing in Chicago suggests that the percentage of students meeting minimum 
competency requirements has increased since the introduction of the policy (Roderick et 
al., 1999). However, the policy has differential impacts on students based on their family 
background characteristics. For example, African American students were retained at a 
much higher rate than their white and Latino/a counterparts because they tend to score 
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lower than whites and because of the higher proportion of Latino/a students who are in 
bilingual programs and therefore exempt from the policy (Roderick et al., 1999). This 
likely has a stratifying effect for African American students because grade retention may 
be associated with negative long-term outcomes including reduction in self-esteem and 
increased likelihood of high school drop-out (Roderick, 1994). 

Muller and Schiller (2000) examine how state-level testing policy impacts high 
school students’ graduation rates and mathematics course-taking. They show that these 
policies equalized students’ academic attainment and reduced the impact of teachers’ 
gatekeeping through low-expectations, seeming to support the arguments of supporters of 
these policies (Muller and Schiller, 2000: 210). However, they also find that when state 
tests link students’ performance to consequences for schools it leads to stratification 
based on SES, lending support to the arguments of opponents of testing policies. 
Therefore, their findings do not strongly support the arguments of proponents of these 
policies or their critics. They recommend more research using both qualitative and 
quantitative methods to explore the mechanisms through which these policies influence 
teachers’ practices and students’ outcomes. Research into the processes of accountability 
policy implementation will help inform this discussion. 

In this paper, we argue that in order to understand the implications of these 
policies it is important to examine how they are understood and implemented in 
particular school contexts. In the current paper, we examine schools’ responses to high 
stakes accountability policy, paying particular attention to the implications of these 
responses for issues of educational equity both within and across institutions. More 
specifically, we examine how teachers and administrators in high and low performing 
schools respond to high-stakes accountability policy focusing on their responses to 
incentive structures, their interpretation and use of test score data, and their subsequent 
instructional priorities. We argue that these responses to high stakes accountability are 
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situated in a school’s status with regard to accountability policy - probation versus high 
performance - and argue that school status is correlated with students’ race and social 
class. We conclude that differences in responses to accountability policy in different 
types of schools may increase rather than reduce educational stratification. 

The paper is organized as follows. First, we outline the theoretical tools used to 
frame our discussion. Following this, we discuss the methodological approach that 
guided the research. We then examine differences in schools’ responses to high stakes 
accountability in four elementary schools - two high performing schools^ and two 
probation schools. In the final section we discuss these cases, paying particular attention 
to the implications of school-level responses to high stakes testing for issues of 
stratification. We argue that the picture emerging from these cases suggests that the ways 
in which these policies are implemented in particular school context may exacerbate 
rather than reduce educational stratification. 

Theoretical Tools 

Research on the role of family background and educational stratification 
demonstrates consistent links between socioeconomic status and students’ outcomes. 
Some explanations for this pattern focus on direct effects of family background such as 
class-based disparities in parents’ beliefs and involvement patterns (Lareau, 1989; Sewell 
and Shah, 1968 (a), Sewell and Shah, 1968 (b); Sewell and Hauser, 1980), family 
structure (i.e. number of parents in the home or the number of siblings in the family), and 
access to extra-familial resources through parents’ social networks and institutional 
affiliations (Coleman, 1988; Coleman andHoffer, 1987; Wong, 1998; Carbonaro, 1998; 
Hao and Bonstead-Bruns, 1998; Hofferth et al., 1998; McNeal, 1999; Wilson, 1987). 
Other scholars focus on the interaction between these background characteristics and 
school practices. In these accounts, schools impact students though micro-political 
processes such as low teacher expectations (Roscigno, 1998; Brophy and Good, 1973; 
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Rist, 1970, 1977; Rosenthal and Jacobson, 1968), tracking (Oakes, 1995), and cultural 
reproduction processes (Bourdieu, 1979; Bourdieu and Passeron, 1977), 

A third approach - institutional stratification- highlights the implications of inter- 
organizational processes for social stratification (Roscigno, 2000), Because race and 
social class shape school attendance patterns and contribute to the creation of highly 
segregated school contexts (Orfield, Bachmeier, James, and Eitle, 1997), family 
background can contribute to stratification through the distinctly different characteristics 
of the schools students attend (Roscigno, 2000), Differences in schools’ monetary 
resources (Elliot, 1998; Greenwald, Hedges, and Laine, 1994; Hedges and Greenwald, 
1996; Kozol, 1991), instructional quality (Smith, Lee, & Newman, 2001), presentation of 
valued knowledge (Anyon, 1991) course offerings (Ayalon, 1994), and social 
organization (Bowles and Gintis, 1976), all contribute to the maintenance of social 
stratification, 

Roscigno’ s institutional stratification perspective captures these inter- 
organizational dynamics, emphasizing the “multi-level and inter-institutional nature of 
racial educational disadvantage” (Roscigno, 2000:271), Roscigno writes: 

Arguably, the most important of these inter-institutional linkages in relation to 
race/class reproduction in education has to do with family background inequalities 
and their consequences for achievement through the character and resources of 
the schools one attends. What this means, more straightforwardly, is that family 
background shapes residential options. Where one resides, in turn, has a large 
impact on the school one attends and, consequently, achievement [emphasis in 
original] (Roscigno, 2000:271), 

He concludes that residential segregation leads to indirect effects of family background 
through enrollment patterns in public versus private schools, the race and social class 
composition of schools, monetary expenditures, and school climate (Roscigno, 2000), 
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Other work demonstrates that the concentration of low-income and African American 



students in certain schools may have detrimental implications for student outcomes apart 
from the individual characteristics of students (Bankston and Caldas, 1996). Taken 
together, this work demonstrates that educational stratification based on race and social 
class is at least partially maintained through race and class-linked institutional processes. 

Building on this work, we argue that the implications of policies like high stakes 
accountability are also shaped by institutional stratification processes. In contemporary 
urban contexts there are several types of schools including private schools (both religious 
and non-religious) which are often thought to be the highest quality, magnet schools 
which are often considered the “elite” public schools, and neighborhood schools which 
can be further divided into high and low quality categories. Social class and race are 
important in patterning the schools that children attend, with the more highly valued 
settings being most accessible to middle- and upper-class children. The different types of 
public schools are likely to implement the policy differently. Therefore, if students are 
concentrated in different types of schools based on race and social class, they will be 
impacted by the policy in distinct ways. 

Taking the Chicago Public Schools as an example, data^ shows that schools on 
academic probation have a higher percentage of African American and low-income 
students, on average, than the typical Chicago Public School. African Americans make 
up 52% of the district’s student population but 83% of students attending probation 
schools. Likewise the district average of low-income students is 84% while the average 
for probation schools has 92% low-income students. Perhaps most interesting, however, 
is the fact that while white students make up 10% of the district student population, they 
make up less than 1% of the students attending elementary schools on probation. 

These figures are more striking when compared to data from Chicago magnet 
schools which consistently rank among the district’s highest performing schools. 
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Elementary magnet schools contain 55% low-income students (compared to 92% for 
probation schools) and 53% Black students (compared to 83% for probation schools). 
White students make up 17% of elementary magnet school students (as compared to less 
than 1% in probation schools).^ These data demonstrate that structural processes related 
to family background shape students’ access to schools of different quality. African 
American and low-income students are more likely to be found in the lowest performing 
schools while white and middle income students are more likely to be found in higher 
performing magnet schools."^ Table 1 presents the mean percentage of students by race 
(black/white) and social class in the Chicago district as a whole as well as its magnet, 
high performing, and probation elementary schools. 

[insert table 1 here] 

The multiple factors that contribute to this process are beyond the scope of this 
paper, however, as we shall see, the observed patterns likely have important 
consequences for students’ educational experiences.^ In the discussion that follows, we 
focus on school status in relation to high stakes accountability - probation versus high 
performance - as an important factor that shapes schools’ responses to accountability 
policy and students’ access to educational equity. 

Research Methodology 

This paper is based on data from the Distributed Leadership Project, a four-year 
longitudinal study of elementary school leadership funded by the National Science 
Foundation and the Spencer Foundation. The project began with a six-month pilot phase 
during the Winter and Spring of 1999 involving seven Chicago elementary schools, four 
interview only sites and three schools where we conducted interviews and extensive 
fieldwork. The first full year of data collection (Phase 1) began in September 1999 and 
focused on eight Chicago elementary schools, two of which were also part of the study’s 
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pilot phase. Year 01 data collection was completed in June 2000, and involved between 
50 and 70 days of fieldwork in each of our eight sites. The four schools that are the focus 
of this paper were selected because two were the highest performing in our sample and 
the other two were probation schools and the lowest performing. 

Site Selection . We used a theoretical sampling strategy (Strauss 1987; Glaser & 
Strauss, 1967) to select study schools based on two primary dimensions. First, the schools 
vary in terms of student demographics, including seven schools that are predominantly 
African American, three that are predominantly Hispanic, and three that are mixed. 
Second, while we are chiefly interested in schools that had shown signs of improving 
mathematics, science, or literacy instruction (in terms of either process or outcome 
measures), we also wanted to study some schools that had managed little change in 
student outcome gains. We focus our analysis in this discussion on two high performing 
schools and two schools that are on academic probation. In table 2 we outline schools by 
racial composition and percentage of low-income students. 

[insert table 2 here] 

Data Collection . Research methodologies include observations and structured and 
semi-structured interviews. In the schools reported on in this paper, researchers observed 
school leadership events, meetings, and classroom instruction in grades 2 and 5 and 
conducted interviews with teachers and school leaders. During Phase 1 of the study, 
researchers spent the equivalent of three to four days per week per school over a ten- 
week period in the Fall of 1999 and a 12 week period in the Spring of 2000. Leadership 
events observed in these schools to date included grade level meetings, faculty meetings, 
school improvement planning meetings, professional development workshops, and 
supervisions of teaching practice. In addition, we observed a number of other events 
including homeroom conversations between teachers, lunchroom conversations, grade 
level meetings and subject specific workshops and meetings. 
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We completed interviews with teachers at the second and fifth grade levels and 
school leaders (including lead teachers). Interview protocols focused on school leaders’ 
agenda and goals, their responsibilities, and the key tasks they perform as part of 
promoting instructional change in mathematics, science and literacy.^ We also selected 
specific instances of school leaders’ practices to observe and then conducted post- 
observation interviews with these leaders about the observed practice.^ 

Using the protocols, researchers wrote detailed fieldnotes following each 
observation. A total of 181 sets of fieldnotes detailing anything from 30 minute meeting 
observations to three hour professional development workshops were compiled thus far. 
All interviews were tape-recorded and transcribed. 

Data Analysis . Data collection and data analysis are closely integrated, allowing 
us to examine patterns and working hypotheses as they emerged from data analysis and 
refine data collection strategies (Miles & Huberman, 1984). Coding categories were 
developed based on the distributed leadership theoretical framework and initial analyses 
of our observation and interview data. A commercial computer based qualitative coding 
program - NUDIST - was used to code all project data. NUDIST allowed us to code the 
emerging ideas and concepts from the data into free nodes that can be compared and 
related to each other, forming larger “parent” nodes that can be stored in an index system 
that brings the different components of the project together. Coders worked together to 
code transcripts initially in order to develop a shared understanding of what each code 
meant. Once coders had developed a “taken as shared” understanding of these codes, 
they worked independently. We also used our field notes (which document the actual 
observed practice of leadership) and interviews to construct our case studies for this 
paper. 
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High Stakes Accountability: The Case of the Chicago Public Schools 



A popular public policy strategy in recent efforts to improve America’s schools 
involves holding school’s accountable for student achievement. Arguing for a K-12 
curriculum that is grounded in more intellectually rigorous content, reformers propose to 
use a variety of policy levers to hold schools accountable for students’ mastery of this 
content. These policy efforts involve at least two components: specific student 
performance outcomes and rewards and sanctions for schools (Clotfelter & Ladd, 1996; 
Elmore, Abelmann, & Fuhrman, 1996). We identify each of these components below 
and use the Chicago Public School’s accountability policy, an often referred to example 
of a successful accountability policy in both policy and academic circles, to illuminate 
each component. 

To begin with, student performance outcomes as measured by tests, rather than 
inputs (e.g., number of certified staff), are the primary mechanism that state and local 
government agencies use to hold school’s accountable. While the 1988 Chicago School 
Reform Act (P.A, 85-1418) included the decentralization of decision making to the 
school site level and the formation of Local School Councils (LSC), the Chicago School 
Reform Amendatory Act of 1995 gave much authority to the chief executive officer, 
appointed by the mayor, who was able to place poorly performing schools in remediation 
or on probation based on their performance on the Iowa Test of Basic Skills (ITBS). 
Specifically, student performance on the ITBS at benchmark grades became the districts’ 
primary measure of school accountability and progress. 

The second component of most accountability measures involve the creation of a 
system of rewards and sanctions as well as intervention strategies designed to motivate 
schools to improve student achievement. In Chicago Public Schools the key sanction is 
the power of the Chief Executive Officer to place schools on probation because of low 
performance as measured by standardized test scores. For example, in 1996 the CEO 
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placed twenty per cent of the elementary schools, 109 schools, on probation because 
fewer than 15% of their students performed at or above national norms on the reading 
and mathematics sections of the Iowa Test of Basic Skills (ITBS) (Hess, 2000; Wong & 
Anagnostopoulos, 1998).^ Schools on probation are required to develop a supplemental 
school improvement plan that outlines specific strategies the school will take to improve 
student achievement and defines criteria that will be used to judge the school’s progress 
toward improvement. For technical assistance, schools use their discretionary funds to 
purchase the services of an external partner whom they can select from a district- 
approved list. If the district decides that a school has not made adequate progress, the 
CEO can have the school reconstituted, ordering new LSC elections and replacing the 
principal and faculty. 

Some efforts to transform accountability arrangements also include rewards or 
sanctions for students. This is important because, as some scholars note, teaching is co- 
produced by teachers and students (Cohen and Ball 1996). Hence, an accountability 
system that targets teachers and school administrators exclusively may place them in an 
impossible position as they depend on their students to improve school performance. In 
1996 the school district also ended social promotion, informing the students that 
beginning with the 1996-97 school year if they failed to achieve at a certain level on the 
ITBS they would have to attend summer school. Further, if by the end of the summer 
students still fail to achieve at the required level on the ITBS they are not promoted to the 
next grade level. These developments are important because the incentive structure 
mobilized by the school district targets students in addition to teachers and 
administrators. 
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General Impacts of High Stakes Testing 



One of the core arguments of proponents of high stakes testing is that it provides 
incentives for school improvement. In simple terms, when schools, teachers, and students 
are made accountable for students’ outcomes, it is argued, they will seek to improve 
them. Data from our schools indicate that school leaders do pay attention to high stakes 
accountability. There were general patterns that cut across all of the schools in our 
sample. Leaders at each school report paying some attention to the exams. Even in the 
highest performing schools, schools where 50% or more of the students score at or above 
national norms in core subject areas, school leaders report paying some attention to exam 
results. In addition, leaders at both high and low-performing schools reported seeking 
improvements in students’ outcomes. As the principal at one of the higher performing 
schools reported: 

When I look at the test results - and I happen to be one that believes that the test 
results do tell you something about curriculum -Fifty percent are succeeding. I 
look at it the other way, fifty percent of our children are not succeeding . . . 
Hopefully our scores will go a notch up. You know, even one percent to show 
that there is some effort and some results. 

The push for improvement was consistent among school leaders across all of our schools. 
None of the school leaders we interviewed was completely content with student 
performance. 

All of the schools also engage in some form of explicit test preparation activities. 
There are differences, however, in the strategies used and the frequency of these 
activities. At one school, which engages in the most extensive test preparation, students 
are tested every Thursday. The stated goal at this school is to make the testing 
environment comfortable for the students so that when they take the actual exam they 
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will not be overwhelmed or feel undue pressure. As the Assistant Principal at this school 
explained: 

We expose them to the testing situation at least once a week and we use the 
bubble forms except for the little ones because their test is not like that. In other 
words we try to simulate the testing even with the writing part of the test. I’ve 
shown the teachers how to do booklets that simulate the real test. So we try to do 
whatever we can so that our children are accustomed to taking tests so they’re test 
smart kids and they’re not nervous. 

Another school used a five-week assessment to structure instructional activities 
for students and target specific problem areas. Finally at other schools these activities 
were more sporadic, occurring increasingly often just prior to the testing period. 

We also found that testing structured schools’ and teachers’ priorities with regard 
to the content covered in classrooms and the attention paid to different subject areas. 

With regard to instructional content teachers reported that testing was outmatched only 
by other teachers and textbooks as an influence on the content they covered.^ One 
specific manifestation of this influence on content coverage was the lack of attention to 
science instruction when compared to reading and mathematics. As a field, elementary 
education is characterized by the differential valuation of mathematics, science, and 
literacy instruction. Although most elementary teachers do not have a well-defined 
subject matter specialization and do not work in situations where organizational 
arrangements (e.g., departmental structures) directly support subject matter identities, 
subject matter is an important context for teachers’ work (Stodolsky, 1988). A general 
pattern at our schools was that science instruction was given lower priority than other 
subject areas. Across our schools we found that when compared to mathematics and 
language arts instruction, science had fewer formal and informal subject matter leaders, 
less attention from school administrators, and fewer instructional specialists (Spillane, 
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Diamond, Walker et al. 2001). Further, our data suggest that accountability policy may 
accentuate this undervaluing of science education. When asked how important testing 
was on the content they covered in science, teachers ranked it below principals, other 
teachers, standards, textbooks, and instructional specialists. Teachers in both high and 
low-performing schools captured the situation: 

So I go to my grade chairperson and she'll give me a list of the ten objectives in reading 
and math that I must teach. Science and social studies are more flexible because the 
students are not tested on the lOWA’s [Iowa Test of Basic Skills] in science and social 
studies so that's more, you know, on the teacher's personal decision. 

You know science isn’t one of your guides for whether a child is promoted or 
graduates. So reading and math are what are stressed because those are what 
everybody looks at. And to a certain degree, that’s what the teachers look at too. You 
know I’ve got to get you on. I’ve got to get you out of this building. You’ve got to 
get this in math, you’ve got to get this ... in reading. So those two always come first. 

We aren’t able to teach science as much as I would like to. Mainly because on the 3'^^ 
grade level we aren’t tested on [science and social studies] we’re not tested on those 
subjects we are tested on reading and math. ... I just can’t fit it in. [There is] so much 
math and so much reading that it’s hard to fit the science and social studies [in]. So 
most of the time ... I begin teaching science and social studies after the test. 

Therefore, across all of our schools, and without regard to performance level, schools 
paid attention to tests results and sought to improve students’ outcomes on them. 

According to teachers’ reports, testing and accountability policy also influenced the 
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instructional content covered and the priority placed on different subject matter areas,, 
with science receiving less attention than reading and math. While general patterns 
existed across the schools, school personnel at high and low performing schools 
responded to these policies differently. The following section discusses some of these 
differences, comparing responses to accountability policy at high and low performing 
schools. 

School Status and The Nature of Incentives 

Proponents of high stakes accountability argue that it will create incentives for 
instructional improvement. We argue that how schools respond to these incentives 
depends at least in part on their status in relationship to accountability policy. In Chicago, 
the incentives for probation schools are clear and direct, they are organized around the 
threat of reconstitution. To avoid this schools must get off of probation and this becomes 
the goal. In high performing schools, the incentives are structured around rewards more 
than sanctions, with recognition for high performance likely being the primary motivator. 
The following section discusses the ways in which school leaders (and schools as a whole 
to a certain extent) at high and low performing schools orient themselves toward 
accountability policy and the ways in which the incentive structures seem to shape these 
orientations. 

Incentives in Probation Schools 

The two probation schools -* Waxton and Field emphasize getting off of 
probation. As Waxton School’s principal stated when asked about the schools goals, “the 
obvious goal is to get off probation! Now that’s it in a nutshell.” This emphasis on getting 
off of probation in these two schools had an important impact on their responses to 
accountability policy. While probation status allows school leaders to demand teachers 
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attention, the need to respond to immediate pressures from external actors such as 
probation managers led to several related responses which tended to be cosmetic and 
superficial with regard to classroom instruction. 

The principal at Waxton School used the accountability policy as a “stick” to get 
teachers’ attention and motivate them. She was in her first year at the school and used 
accountability policy as a way to legitimize her push for teachers to improve. She 
reminded teachers that she would replace those who were not pulling their weight, 
suggesting that this was a “requirement” she would have to reluctantly enforce. In one 
staff meeting, for example, she told teachers about how she had been asked by the 
probation manager (and the central office) to begin the process of identifying teachers 
who might potentially be replaced. She explained to the teachers that, although she 
“could no longer ignore” these requests by the probation manager, she would drag the 
process along to give each one of them “a chance” to improve. This was a tone she 
consistently sounded, seeking to motivate teachers. In this sense, accountability policy 
(and the set of actors associated with it) serves as a threat, a way for school leaders to get 
the attention of teachers and push them to change their practices. 

The external partner at Field School believes that teachers at this school are open to 
change. As she has observed at this school and at others “teachers in probation schools 
are more likely to be open [to change]. They know the next step is possibly 
reconstitution.” Therefore, at both of the probation schools there is a sense that 
accountability policy can be use to motivate teachers to be responsive to the efforts of 
school leaders. 

Another motivation strategy used at Waxton school was the pep-rally strategy. Here, 
leaders sought to encourage teacher effort by expressing their confidence in their abilities 
and promising that with hard work they could get off of probation. Consider the first 
meeting of Waxton’s staff with the school’s new probation manager: 
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There was clapping of hands as Beatrice stood up to speak. She began by saying "it 
is possible to get off probation" (another loud applause) and "you are going to get off 
probation." (another loud applause). Beatrice then gave a little talk about how she 
thinks the staff and children at Waxton are capable and can achieve. "Sometimes", 
she continued, “it’s only a question of knowing what to focus on, getting children to 
be ready for the tests and getting the right tools.” 

There was an air of a revival meeting atmosphere in this meeting, the emphasis being 
liberation from the bondage of probation. This revival meeting approach was evident 
again at another meeting: 

Beatrice quickly read off the next activities on her list (noting that they were 
running out of time) "Follow up in the classrooms after discussing each 
strategy," "test-taking - we will do breakdown of test skills, so you focus on the 
right skills that are asked in the tests" "We did this at [Maxwell] School and 
found it very helpful" (“yessess” from teachers and interjections of "that's what 
we need!"). Beatrice’s colleague emphasized that Waxton will "get off probation" 
which was followed by applause. 

In a certain sense this strategy dovetails well with the use of probation as a stick to 
motivate teachers. Being placed on probation is likely depressing. It labels the school as a 
failure and places it in jeopardy of being “reconstituted.” In addition to this if school 
leaders work too hard to challenge teachers without some sense of emotional support, it is 
likely that it will be difficult to make change. This pep rally/revival meeting approach 
may be well suited for creating social support in the push for academic improvement. 

At the probation schools, accountability policy, and particularly probation status, 
enhanced school leaders’ ability to get teachers’ attention. This is a potentially powerful 
tool in the effort to implement instructional change, a tool that proponents of high stakes 
accountability point to in support of these policies. 
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Managing the Impressions of External Actors 

Another set of incentives that existed in probation schools but not high 
performing schools was the need to impress external observers. Probation schools must 
have an external partner and a probation manager. These actors play a role in determining 
whether or not a school gets off of probation. Some probation school efforts were 
designed to convince external observers that their schools were doing all they could to 
improve students outcomes. The pressure to do this was captured by the principal who 
relayed a conversation she had had from an office of accountability representative. 

When Dr. Austin came out yesterday, he’s . . . from the office of accountability. 

He said, “You know there are schools that are in worse condition, physical 
condition than your school. Those schools are open because they are at 40 
percentile.” And he said, “We don’t care if they run naked through the hallways 
but they’re not on probation. Whatever it is that you need to do then that’s what 
you will have to do to get your school off of probation. So there’s still something 
wrong that has to be fixed ... So whatever it takes, you have to fix it.” 

At Field school, some of these responses emphasized superficial changes 
designed to impress district officials and probation managers. For example, since being 
placed on probation. Field school has adopted several programs in a rather haphazard 
way in an attempt to demonstrate improvement efforts. This approach has been 
unsuccessful in the eyes of one of the schools external partners, a consultant from a local 
university, who argued that the problems of the school result from a lack of coherence in 
their improvement efforts. Likewise, the following field note expresses a similar sense of 
the lack of coherence of the schools program. 
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After sitting through a number of ... meetings, I’m still struggling to see 
coherence to the schools struggle to get off probation. While the components 
seem to be in place ... the quality of those components seems to be lacking. 
Schools adopting several disconnected programs have been characterized as “Christmas 
tree” schools. This pattern surfaced at many unsuccessful elementary schools 
(Consortium 1993). In contrast, more recent work suggests that increasing instructional 
program coherence is an important mechanism for instructional improvement (Newman, 
Smith, Allensworth, and Bryk 2001). 

Another strategy was to impress external observers through emphasizing the 
appearance of student and teacher engagement in instructional activity and student 
discipline. For example, at a Field School staff meeting the assistant principal warned 
teachers that the school’s probation manager would be visiting and advised teachers on 
what actions to take. 

I don’t know if you saw Ms. Fox, our probation manager, here in the building 
today. ... Make sure your university organizers are visible, there should be 
student work, your classroom should be attractive, they are going to be looking at 
the decorum of our students so make sure you continue with working on good 
behavior, most important they are looking for students on task and that teachers 
are effectively teaching their students. . .there is no telling when someone might be 
in to visit your class. 

This reflects one of Field School’s responses to external pressures. The emphasis here is 
on classroom management, decor, and the displaying of students work. The principal 
does discuss the importance of demonstrating being “on task” but focuses less on 
instruction issues and more on classroom management and appearance. In other words, 
this strategy is cosmetic, emphasizing the trappings of instructional improvement while 
the actual instructional practices are not emphasized. 
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Responses to high stakes accountability in the probation schools naturally emphasized 
getting off of probation. The strategies used by school leaders included using the school’s 
probationary status to motivate teachers - both through the threat of negative 
consequences for low-performing teachers and through the promise of getting to the 
probation free “promised land” and efforts to manage the impressions of external 
stakeholders through adopting multiple programs and increasing the appearance of 
instructional emphasis and innovation. 

Incentives in high performing schools 

Incentives in the high performing schools are less clear than incentives at 
probation schools. One strategy used by school leaders in these schools revolved around 
praise for accomplishment. In Kelly School professional development meetings the 
principal regularly praised teachers for working hard to produce high student outcomes. 
Teachers were encouraged to applaud their own and other accomplishments and test 
results were prominently posted inside the school and shared with parents who visited the 
school. All of this reinforced pride in past accomplishments and encouraged continued 
improvement. 

Given past success however, leaders were also forced to combat complacency on 
the part of faculty. In the case of Baxter, the principal felt that teachers believed that the 
school was performing better than it was. He used trend test-score data to challenge 
teachers assumptions. He demonstrated that while the schools absolute outcomes were 
excellent, when compared to other schools in their neighborhood their students were not 
gaining as much year to year. As he explained: 

The analysis made clear that out of the 12 schools [in the neighborhood], Baxter 
was either at the bottom or really close to the bottom, in terms of the amount of 
actual growth that students were making. Forget about where the growth started, 
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forget about the base. Forget about the end. Just, you know, how many months of 
progress, on an average were sixth graders achieving. 

School leaders used this data in school meetings to help create incentives for continued 
instructional improvement. They took great pains to repackage standardized test data in 
ways that captured teachers’ attention, transforming massive spreadsheets into relatively 
easy to read charts that were color-coded by grade-level. 

Likewise, at Kelly School, a monitoring system with a template that connects 
teachers daily lesson plans to the material tested, district standards, and the level of 
mastery of specific skills for each student was used to maintain a focus on constant 
improvement. The assistant principal explained how this skill chart works. 

It’s just an organizational tool. You look at this chart and you see that child 
didn’t master that skill. That is something you can do in a small group. You can 
assign your [teacher’s] aide to work with that particular child on that skill and 
retest, cause we believe right away if the child didn’t master it . . . most kids only 
miss it like a master is 80% maybe they got a 75 or a 72, so he just missed it by a 
little bit, quickly review, go over it again and retest. The child masters it then 
move him on. 

At a professional development meeting the principal emphasized that teachers need to 
insure that they are tracking the skill mastery of students. 

“I noticed that the skill charts are not being filled out diligently enough. The 
[university program] got us on the path to improvement through charting our 
progress but we can’t get lax on this. We have completed 2 fifth week 
assessments and if you have a lot of children not getting their skills you need to 
re-teach. If a lot of your children are not getting the material it is not the children. 
It is something to do with the way you taught it. You can’t teach the same way 
every year. Its always the children. People make excuses. But that does not hold 
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up because we can take the same child in two different classes and they can do 
well in one and have trouble in the other. But if you see students are having 
trouble don’t go on. Its going to be evident that the students are not getting it so 
think of another way to teach it. 

Here school leaders help create a school-based incentive structure tied to the maintenance 
of high performance. In this example, the teachers maintain primary responsibility for 
student performance and excuses based on students’ limitations are discounted. In 
another meeting (the following week) she again emphasizes the need to maintain focus 
and work hard in order to keep test scores up. 

[The principal] said that someone inquired about Kelly and asked if the school has 
gifted students. She said that “our students are average . . . our instructional 
program is what makes the difference. . . the only way we continue to improve is 
through hard work. Just because we did well last year [on the ITBS] does not 
mean anything. We have to continue to work hard and align our lesson plans. 

In high performing schools leaders used praise for past performance and the need for 
constant improvement to heighten teachers’ sense of accountability and motivate them. 

In contrast to probation schools, the incentives in high performing schools were 
based more on rewards than sanctions. In these schools, school leaders praised school 
accomplishments in professional development meetings and proudly displayed students’ 
outcomes in the school and communicated these to parents. In addition, because the 
accountability policy did not create sanctions for these schools, school leaders sought to 
interject other forms of motivation through comparisons with other schools or through 
building on the school’s past performance. 

Therefore the nature of the incentives created by high stakes accountability are 
different for high and low-performing schools. High performing schools are arguably in a 
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better position to focus on instructional improvement while probation schools must 
respond directly to external pressures while they seek to enhance students’ outcomes. 

Accountability Policy and “Objective” information 

Proponents of accountability policy also argue that test results provide objective 
information upon which to orient school decision making. The argument is that schools 
with clear, objective information about student performance will be able to make better 
informed decisions about instructional improvement (Coleman 1997). Once again our 
data demonstrate distinct patterns in the interpretation and use of results in high 
performing versus probation schools. We argue that the ways in which test results are 
interpreted depends, at least in part, on the school context. 

Interpretation and Use of Data at High Performing Schools 

At both high performing schools, the full range of test score data was used to 
inform strategies of instructional improvement. At both schools, leaders use the outcomes 
for the entire school in addition to the item analysis to track overall school trends. For 
example, Baxter school’s principal and a collection of school leaders^® are involved in 
data interpretation including longitudinal trend analysis and the analysis of movement 
between quartiles. During an initial meeting with the principal, he shared a longitudinal 
data analysis for both mathematics and reading which highlighted the movement of 
students between quartiles for the entire school and for specific grade levels. In addition, 
he reported on the development of a particular grade level, the class of 2003, in terms of 
their performance in mathematics and reading. 

At both Kelly and Baxter school leaders use the item analysis to identify specific 
student needs within subject areas. For example, the Kelly principal explained how the 
item analysis allowed school leaders to identify overall student needs: 
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We try to look at [test scores] in August if we have them back and we design our 
program a lot looking at those skills that are measured on the ITBS [Iowa Test of 
Basic Skills] and what the item analysis^^ indicate our weaknesses and our 
strengths and so forth. 

Kelly’s principal explained how the school uses test scores to track student performance. 
When asked about how this information informed the instructional program in 
mathematics the principal explained.^^ 

With the math I found that concepts, our children tend to do well in computation, 
pencil and paper, figuring out the problem, 2 + 2, whatever they do well but when 

it comes down to the concept and which looking hopefully they’re using - they 

needed to use higher order thinking skills they tend to not do as well and we’re 
working - we started last year we started focusing in on higher order thinking 

skills because moving more and more in that direction. The math problems 

they have to explain how they got the answers not just get the answer. So what 
we’ve been doing we have been working with this year a new initiative not all 
teachers are doing this but next year every teacher will be doing this. The math 
journal in which children they must explain - they must explain how it is that they 
arrived at the answer that they got and as I said the total math is not - it’s pretty 
good cause we’re at 61% but the math problem solving we tend to not do as well 
and especially in the area of math concepts. So we’re working on that. 

The principal’s interpretation is enhanced through her interactions with others in the 
school who inform her interpretations and seek to implement instructional responses. The 
school’s assistant principal, counselor, and technology coordinator aU play active roles in 
data interpretation. The Technology coordinator identified similar interpretation of the 
test score data. 
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The biggest deficit in mathematics grade-wise, and we found it to be pretty much 
the same in every grade, was the word problems. Them interpreting word 
problems. Their computation skills are great. Just about in every grade. The 
computation scores were good, average or above average. Data interpretation was 
another area that we felt needed some work. 

Baxter made similar use of the test score data. As the principal at this school explained: 
One thing that helped us a lot in terms of being able to disaggregate our math data 
is we were able to disaggregate it into three sections called problem solving and 
data interpretation, concepts and estimation, and computation. 

Therefore, both of the high performing schools used test results to identify macro-trends 
across the school and focus attention on areas of specific needs. In both schools, the 
“item analysis” was a tool that helped them identify where they should focus their 
attention. 

Interpretation and Use of Data in Probation Schools 

At the probation schools the interpretation and use of the data was more general. 
School leaders discussed the need to improve reading and mathematics but did not speak 
in specific terms as they did in the high performing schools. There was limited discussion 
of sub-dimensions of subject matter areas, rather the discussion revolved around reaching 
the probation threshold in the two primary subject areas. In addition, the data was used at 
these schools in pretty much the form in which it came from the district. There was less 
re-packaging of the information and limited analysis of specific tends (with the exception 
of the identification of specific students who were close to “passing” the exam). 

The high performing schools use the test score data in ways that seem consistent 
with the arguments of testing proponents. The test results are used to define students’ 
specific instructional needs and provide a basis for school level instructional decision 
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making. In contrast, the probation schools tend to focus more on the overall test results 
and have a less systematic strategy for turning test results into useful information for 
instructional change. This suggests that even the interpretation of “objective” data is 
situated. Within higher performing schools, there is a more substantive interpretation of 
the data. It suggests that the resources for data interpretation are greater in higher 
performing schools. Therefore these schools are more likely to benefit from the 
information in ways that lead to instructional improvement.^^ 

Accountability Policy and Academic Press 

Academic press measures the normative emphasis on academic success and 
reaching certain standards of achievement among both teachers and students (Lee, Smith, 
Perry, and Smylie 1999). We argue that instructional focus is likely to increase in these 
schools with high-stakes accountability. Schools will focus on academics more when the 
incentives structure shifts, with rewards and sanctions being tied to students performance. 
However, academic press as typically measured is content neutral. It suggests that high 
standards exist but tells us little about how these standards are operationalized inside 
schools. Therefore, what academic press means for different schools may vary. 

Moreover, when schools are positioned differently within the accountability system their 
“press” might manifest in different ways. We found that the probation schools increased 
instruction focus, but in ways that were designed to respond to the policy demands of the 
external environment - getting off of probation. Their efforts targeted certain students, 
certain grade levels, and certain academic subjects. In contrast, the high performing 
schools focused equally on mathematics and language arts instruction, emphasized 
improvement for all grade levels, and worked to enhance the learning opportunities of all 
students. The following section discusses these patterns. 
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Targeted Instructional Focus in Probation Schools 

One of the critical issues that was continually raised at the probation schools was 
getting off of probation. One way that the instructional focus manifested at these schools 
is in an effort to increase the number of students at or above cutoff points at benchmark 
grades. In these approaches, school leaders target certain students or certain grade levels 
for extra assistance in an attempt to reach minimum acceptable performance levels. For 
example, at Waxton school teachers and administrators focus on the benchmark grades in 
order to reduce their retention rates. At Waxton, 50% of the eight professional 
development meetings we observed over the year were largely or entirely focused on 
some aspect of testing including topics such as skills tested in language arts; skills tested 
in mathematics; constructing multiple test items; and preparing students for the ITBS. 
Professional development and other efforts to improve testing, however, tended to be 
targeted to particular grades (those that took the test) and subject areas. The external 
partner focused its energy on teachers in the benchmark testing grades and provided 
exam preparation books only for teachers at those grade levels. 

At Field school, one approach was to identify those students who were close to 
reaching national norms, and providing them with additional help. The school established 
an after-school tutoring program for these students. This tutoring process and the student 
selection process was discussed at a staff meeting by the assistant principal. 

The after school program will start on Tuesday. All of you got the applications 
for your children and they need to be returned on Monday ... we have one class 
for every grade level. The list of students may have seemed erratic. Ms. 
Lawrence chose those students according to their ITBS scores. She chose those 
students who she felt had the most potential to improve. 

Later, a school administrator explained that the school targets those students who are 
closest to the threshold in its effort to get off of probation. 
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The targeted assistance program is . . . for students who are . . . very dose to 
having the skills necessary to pass the test. ... students in this program attend 
three times per week from 2:30 (dismissal) until 4:30pm. Other students are 
allowed to come but the students who are closest to passing the exam are targeted. 
The Assistant Principal later explained this further stating that the school will use this 
program to: 

Work with selected students, taking a look at the IOWA scores and just really 
focusing in on students that are at a median that we can work with and see if we 
can get some growth spurts on them. 

The school external partner, who attributed this focus to the external pressure being 
applied by the school district, also discussed this strategy. 

They [the school] leave behind [lowest performing students] and focus on [the 
higher performing]. So many principals are under this pressure. It’s the name of 
the game. When Valias [the district CEO] comes and they have their region 
meetings, they are told they have to get off probation. Even if your school shows 
growth and doesn’t get off probation, they realize they will be looked at as not 
doing the job. 

Thus the lowest performing students in this school received limited assistance in 
improving their scores. Instead, the instructional focus is on the students who were close 
to making the cut-off for probation requirements. In this case the external accountability 
mechanisms lead to a selective increase in instructional focus with limited implications 
for the lowest performing students. In addition to this selective focus, the program 
emphasizes content coverage (pacing) but not the teaching strategies that should be used. 

Responses to testing in the probation schools were also structured by subject 
matter. As we noted above all of the schools focused more on mathematics and reading 
instruction than science instruction. However, in the probation schools, the instructional 
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focus emphasized one subject area - reading. For example, Wax ton’s administration was 
putting much of their efforts into language arts as opposed to other subject areas. The 
Waxton principal explained the school emphasis on language arts instruction: 

Being very honest, language arts specifically reading is one area that could impact 
probation and since the school had been on probation for so long we felt a need to 
address that curriculum area. And the mathematics scores were slightly higher than 
the reading so that gave us the second reason. 

While instructional emphasis is one of the outcomes predicted by proponents of testing 
policy, the selective emphasis on certain students, grade levels, and subject matter areas 
may limit the impact of the policy for all of the students in the school. Moreover, the 
selective targeting of students seems to run counter to the intended impact of the policy. 
Students who face challenges may in fact be marginalized by the responses of school 
leaders in low performing schools. 

Testing and Instructional Focus in High Performing Schools 

In both high performing schools the exams are used to identify high and low- 
performing groups of students. However, in contrast to the probation schools, the high 
performing schools adopted interventions for all students, not Just a sub-category. At 
Baxter school, test score data was used by school leaders and teachers to diagnose the 
effectiveness of certain teaching approaches. For example, when 5^*^ grade teachers at 
Baxter met to discus the prior years test results, they were pleased that the overall 
percentage of students at or above national norms had increased. However, upon closer 
examination, they discovered that the increases were among students moving from the 
second to the third quartile. In discussing this, they determined that as a group it was 
likely that they had been focusing instruction on students in the middle range, potentially 
not addressing the needs of the lowest and highest performing students. As Baxter’s 
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principal explained, while the school advocates heterogeneous grouping the teachers had 
identified a potential pitfall. 

If you’re go and you look at the data that’s out there about how student grouping 
effects achievement and you recognize that heterogeneous grouping with a 
competent teacher is gonna be your best shot at being able to give everybody an 
opportunity to succeed. What’s the biggest challenge you then have as a teacher? 
Well the challenge is how do you deal with that? How do you manage that 
enormous diversity of talent that’s in front of you? . . . How do you do that? 

That is to say you know the moment that you allow yourself as a teacher to either 
shoot at the middle you’re gonna be, and under-serve the youngsters who are 
really more ready. Or even worse, that you out of mostly good intentions to drive 
most of your teaching efforts by those youngsters who require the most 
remediation and are most needy of your time and who you’re feeling most guilty 
because you’re not serving. 

Given their review of the test score data at the fifth grade level and similar analyses at 
other grade levels, the school developed an approach that sought to address the needs of 
all of the students while maintaining heterogeneous grouping. 

Well, our assumption is that in order to be able to [address the needs of all 
students] you need to teach high, if I can use that sort of piece of jargon, you need 
to teach high and re-teach to the middle and lower. ... the strategy at the school is 
to continue to do the whole group instruction high. Teaching you know the mass 
majority of that whole group instruction is to teach high and then to make 
remedial provisions within the classroom as well as out of the classroom for 
additional tutorial and other kind of remedial work to get to those youngsters who 
are not getting it the first time. That’s the strategy. 
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At Baxter the analysis of movement within quartiles, and teachers’ and school leaders’ 
interpretation of that data, highlighted an instructional practice that needed to be revised. 
Test results therefore informed teaching strategies in ways that targeted the instructional 
focus toward all students rather than a sub-category. 

The instructional focus of the high performing schools also extended beyond the 
benchmark grades to include all grade levels. At Baxter, school leaders tracked test score 
trends for all students and focused attention to all grade levels. For example, spearheaded 
by the principal, Baxter’s leadership committee undertook careful study of school 
standardized test data in math. They wanted to know how and whether high math scores 
in grade 3 were being sustained or not through grade 5. Through longitudinal analysis, 
the team determined that somewhere in grade 6, students’ scores started to slump. 
Suspending school convention, the leadership committee convened a joint task force of 
two groups that traditionally had little interaction: the third and fourth grade teachers and 
the fifth and sixth grade teachers. This joint committee met for the good part of a year 
and as a result of their work tried to build greater alignment in math topic coverage across 
grade levels. Therefore test data provides an important resource for planning at this 
school with the instructional focus extending to all grade levels rather than a subset. 

Kelly school used the monitoring system discussed above to target instruction to all 
students and all grade levels in ways similar to Baxter School. 

Testing resulted in very different patterns of instructional focus at the two sets of 
schools. In the probation schools, the instructional focus became targeted at certain grade 
levels, certain students, and certain subject areas. Low-performing students and students 
at non-benchmark grades were unlikely to be impacted by these strategies. In contrast, 
the high performing schools targeted instruction at all grade levels and all students. 
Though not reported in the data here, they also maintained a balance between 
mathematics and literacy instruction. 
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Discussion and Conclusion 



Having examined schools’ responses to high stakes accountability policy in high 
performing schools and probation schools we argue that these contexts impact how the 
policy is enacted. Proponents of these policies argue that they will create new incentives 
for teachers, provide objective information for school decision making, increase 
academic press, and that the combination of these mechanisms will reduce gatekeeping 
practices. Our data suggest that, across each dimension, leaders at probation schools and 
high performing schools structured their responses differently. 

In probation schools, responses to high stakes accountability emphasize getting 
off of probation, partially through managing the impressions of external stakeholders. 
These efforts to convince outsiders that the school was making change efforts perverted 
the intention of the policy in certain circumstances, prompting an emphasis on the 
trappings of instructional innovation rather than substantive change. In addition, leaders 
at these schools used test results to look at overall school and grade level outcomes but do 
not connect these results explicitly to instructional decisions as supporters of these 
policies suggest they would. Finally, with regard to increased instructional focus 
(academic press), while school leaders were able to demand teachers attention through a 
combination of threat and encouragement, they focused this attention in a process best 
described as “selective press” which targeted specific grade-levels, students, and subject 
matter areas all emphasizing reaching the goal of each school - getting off of probation. 

In contrast, the incentive structure at high performing schools pushed school 
leaders to reward and encourage teachers for their accomplishments while creatively 
pushing for continued improvement. These schools use test data to track macro processes 
of student performance and set the schools’ instructional agenda. This instructional 
agenda, unlike at the probation schools, focused on all students, all grade levels, and 
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balanced attention to both of the core subject areas. Therefore, some of the key 
mechanisms through which high stakes accountability are supposed to impact students’ 
educational opportunities are constructed very differently depending on the school’s 
status in relation to the accountability system. This is very important considering that 
students race and class often play a role in the type of school they attend. 

Proponents of high stakes accountability suggest that these policies will reduce 
schools’ gatekeeping processes and increase academic press. The data from these schools 
suggests that the extent to which this is the case may depend on the status of the schools 
in relationship to the high stakes accountability policy. The data from Field and Waxton 
demonstrate that their response to accountability policy is situated in their probation 
status which structures school-level responses to high stakes accountability in important 
ways. Because probation status adds pressure to school leaders and teachers, it can lead to 
practices that increase rather than reduce gatekeeping processes. For example. Field 
School’s practice of focusing tutoring programs to serve only the students who are close 
to national norms may have detrimental impacts on the lowest performing students, those 
who the policy is designed to help. These students may be marginalized from 
interventions that could increase their educational outcomes. Moreover, the targeted 
responses of these schools, focusing on benchmark grades and subject areas with the 
greatest chance of passing the probation threshold seems equally problematic. Focusing 
on the benchmark grades for intervention suggests that the increase of academic press 
may only impact a sub-set of students. Likewise, if the subject matter areas are 
selectively focused upon this may ultimately limit students access to knowledge. The 
responses of Waxton and Field schools do not seem to represent an increase in academic 
press as much as a calculated, strategic effort to respond to the policy demands of the 
external environment which may ultimately marginalize the lowest performing students. 
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Unlike the probation schools, the responses of school leaders at high performing 
schools seem to more closely match the outcomes predicted by testing supporters. They 
contribute to the development of clearer instructional foci and seem to reduce 
gatekeeping processes within these schools by focusing on improving the learning 
opportunities for all students across all grade levels. 

The responses of these schools suggest that the implementation of accountability 
policy may work against increased educational equality. If higher performing schools 
construct the policies in ways that increase their academic press and reduce gatekeeping 
while lower performing schools have the opposite effect, focusing primarily on 
responding to external threats, then the policies could exacerbate rather than challenge 
educational stratification. The situated nature of policy implementation should be an 
important consideration for school reformers. Policy implementation is very much a local 
process and understanding the variation in context even within districts appears to be 
critical. Moreover, the fact that the likelihood of attending schools of different quality is 
associated with social class, race, and residential segregation suggests that broader 
structural factors impact school level processes in ways that should be attended to by 
policy makers. 

This paper is not meant to argue against high-stakes accountability. In reality, 
these policies have led to increases in students test results across the Chicago district and 
show promise for increasing schools’ focus on instruction, even if this happens in 
imperfect ways. However, this paper is meant to sound a cautionary note with regard to 
the reduction of race and class stratification. The findings reported here suggest that the 
highest performing schools, those with higher percentages of middle-income and white 
students, may benefit more from accountability policy than the probation schools that are 
most in need of improvement. While this is clearly not the intent of the policy, attention 
should be given to the potential implications of this process. 
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Table 1. 

Mean Percentage of Students in CPS Probation, Magnet and High Performing Schools 

by Race and Social Class 





District 


Probation 


Magnet 


High Performing 


% African 
American 


52% 


83% 


53% 


27% 


% White 


10% 


.12% 


17% 


34% 


% Low Income 


84% 


92% 


55% 


56% 



Table 2. 

School Demographics 



School Name 


Racial Composition 


% Low-income 


Baxter 

(High Performing) 


40% White 
6% Black 
26% Hispanic 
26% Asian 


63% 


Field 

(Probation) 


100% African American 


99% 


Kelly 

(High Performing) 


100% African American 


85% 


Waxton 

(Probation) 


100% African American 


97% 
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Icago Public Schools Student Population by Percent 

Low-Income 



i 






Percent Non-Low- 
Income 
16 % 




Percent Low- 
Income 
84 % 



2 




51 



Percentage of Low-Income Students in Chicago 
Probation Elementary Schools 



Percent Non-Low- 
Income 
8 % 




Percent Low- 



Income 

92% 
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NOTES 



We define high performing schools as schools where 50% or more of the students perform at or above 
national norms on the Iowa Test of Basic Skills. The Chicago Public Schools define probation schools as 
those in which fewer than 15% of students perform at or above national norms on the ITBS. Appendix A 
summarizes the data on student race and social class characteristics in the Chicago Public Schools as a 
whole, elementary magnet schools, and probation elementary schools. While we focus on the data from 
these four schools, our data base consists of the 13 schools that are part of the Distributed Leadership 

Study. 

2 

The data reported here is taken from the Chicago Public Schools Office of Accountability public use data 
base. This information can be accessed through this site at http://acct.multil.cps.kl2.il.us/ 

3 

Appendix A contains a series of charts showing these patterns. 

4 

We do not suggest that family background is an absolute determinant of the type of schools children 
attend. A substantial number of the students in magnet schools for example are African American and more 
than half are low-income. However, the data demonstrates a pattern of students being sorted into different 
types of schools in ways that correlate with race and class. 

5 

For those interest in school choice and parents decision making around school attendance please refer to 

Fuller, Elmore and Orfield (1996) and Gewirtz, Ball, and Bo we (1997). 

6 

Our interview questions were designed to get at five core issues about the practice of leadership: 

a. Getting the leaders to identify the key goals or macro functions they work on (e.g. building a school 
vision, promoting teacher professional development, improving test scores, etc.) 

b. Getting them to describe what day-to-day tasks they perform to attain these goals, i.e. the micro tasks 
(e.g. observing classrooms, forming breakfast clubs, facilitating grade level meetings, etc.) 

c. Getting them to describe how they enact the micro tasks; that is their practice as leaders. 

d. Whether and how macro goals/functions and micro tasks are co-enacted; i.e. the extent to which their 
functions are executed with the help from others in the school. 

e. What tools and material resources (including designed artifacts, memos, protocols, organizational 
structures) the interviewees identified as important in the execution of macro and micro tasks. 

7 

Observation protocols focused on: 

1 . The nature and substance of the task: what the leader(s) did and the goals of the activities including 
the subject matter focus of the activity, if any. 

2. How the task was enacted: including the artifacts/materials used and how they were used to enable 
practice. 

3. The timing and location of the task: the physical setting and context of the enactment, and the time 
of the year, week, or day on which the task was enacted. 

The patterns of involvement: including what the leaders/ facilitators did during the enactment, whether 

leadership was shared or not, and role of participants. 

8 

This percentage of students at or above national norms has increased incrementally since that time. 

9 

The categories of influence included other teachers, principals, assistant principals, the local school 

council, parents, testing, standards, textbooks, and instructional specialists (i.e. mathematics teachers). 

10 

For example, in the case of a literacy committee report which drew extensively on test results, the 
Russian Bilingual teacher, a reading teacher, the librarian, the reading specialist, the drama teacher, and a 
local school council representative were all involved in the interpretation of test results and the 

development of the report. 

11 

The item analysis is a document that shows classroom and student level test scores by the items correct 

and incorrect. 

12 

The principal had a similar response for language arts. We limit the discussion to mathematics in order to 
save space. 

13 

While they may not gain as much in terms of absolute test score results, this likely occurs because of the 
fact that gains are more difficult at the top end because of ceiling effects. 



6 



55 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 




Educotiooal Resources Inlormotion Center 



UD O'bsO^^ 



I. DOCUMENT IDENTIFICATION: 



Title; High ShlEeS AcCOuAUbiUlt^ l A 


Author(s); , \n 


llA B g 7^ *^pi| 




Corporate Source: 

Mshfcle 'fs/ H\ 


Morji 


\\Aie^(A (]. 


Publication Date: 

Zoo'Z. 



II. REPRODUCTION RELEASE: 



In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy, and 
electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if reproduction 
release is granted, one of the following notices is affixed to each document. 

If permission is granted to reproduce and disseminate the identified documents, please CHECK ONE of the following three options and sign at the bottom 
of the page. 



The sample sticker shown below will be 
affixed to all Level 1 documents 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



The sample sticker shown below will be The sample sticker shown below will be 

affixed to all Level 2A documents ■ affixed to ell Level 2B documents 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE. AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY, 
HAS BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTgp BY 








TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

2A 


^^0 THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

2B 



Level 1 



Level 2 A 

□ 



Level 2B 




Check here for Level 1 release, permitting reproduction 
and dissemination in microfiche or other ERIC archival 
media (e.g., electronic) and paper copy. 



Check here for Level 2A release, permitting reproduction Check here for Level 28 release, permitting reproduction 

and dissemination in microfiche and in electronic media for and dissemination in microfiche only 

ERIC archival collection subscribers only 



Documents will be processed as indicated provided reproduction quality permits. 

If permission to reproduce is granted, but no box is checked, documents will be processed at Level 1. 



Sign 
here, ^ 
please 



/ hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate these documents 
as indicated above. Reproduction from the ERIC microfiche or electronic media by persons other than ERIC employees and its system 
contractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other service agencies 
to satisfy information needs of educators in response to discrete inquiries. 




Printed Name/Position/Title: 

Tl 'RaEse /'biY. 


Organization/Address: 














o 

ERIC 



III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of these documents from another source, 
please provide the following information regarding the availability of these documents. (ERIC will not announce a document unless it is 
publicly available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are 
significantly more stringent for documents that cannot be made available through EDRS.) 



Publisher/Distributor: 



Address: 



Price: 



IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 
address: 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse: ERIC Clearinghouse on Urban Education 

Teachers College Box 40, Columbia University 
525 West 120^^ Street 
New York, New York 10027-6696 



EFF-088 (Rev. 2/2000) 



